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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)^ Responsive to communication(s) filed on 30 October 2007 , 
2a)n This action is FINAL. 2b)K This action is non-final. 

3) n Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim{s) 1 and 4 - 32 is/are pending in the application. 

4a) Of the above claim(s) 11 - 13, 15- 21, 27-29. 31. and 32 is/are withdrawn from consideration. 

5) n Claim(s) is/are allowed. 

6) S Claim(s) 1, 4- 10, 14, 22-26. and 30 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) n Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) P The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 16 November 2000 is/are: a)S accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)IElAII b)n Some * c)n None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Application/Control Number: 
09/712,925 
Art Unit: 2622 

Continued Examination Under 37 CFR 1J14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1 .17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on October 30, 2007 has been entered. 

Response to Arguments 

2. Applicant's arguments, see page 16, filed October 30, 2007, with respect to the rejection 
of Claims 1 and 22 under 35 U.S.C. 103(a) as being unpatentable over Lyons in view of 
Anderson have been fully considered and are persuasive. Therefore, the rejection has been 
withdrawn. However, upon fiarther consideration, a new ground of rejection is made in view of a 
new interpretation of Lyons. 

3. Specifically, Lyons' figure 9 shows the inputting of parallactic images, as described by 
Applicant on pages 15 - 17 of the remarks. The Examiner previously relied upon the tracking 
system shovra in Lyons' figure 1 . The tracking system of figure 9 shows a narrow angle camera 
(51) and a wide angle camera (51) both attached to a PT base (56) in different locations but are 
both aimed in a same spot to capture images of an object (see column 10, lines 63 - 66). The 
images captured by these cameras are input to the tracking system as parallactic images. In other 
words, the images have parallax with respect to each other. 
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Claim Rejections - 35 USC §103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 1, 5 - 10, 14, 22 - 26, and 30 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Lyons (US 6 734 91 1 Bl) in view of Anderson (US 7 106 376 Bl). 

The Examiner believes independent Claims 1 and 22 are corresponding apparatus and 
method claims, respectively. Accordingly, they will be rejected together. 

6. For Claims 1 and 22, Lyons discloses, as shown in figure 9, an image processing 
apparatus (figure 9), comprising: 

an image data input unit (55) for inputting image data of a plurality of objects (21,21a, 
22, and 23 - see figures 2A and 2B) including a main subject (21); 

a range setting unit (55) for restricting a searching range in the depth direction (either 
"wide-angle field of view" or "narrow-angle field of view"; see column 10, lines 46 - 62 and 
column 11, lines 1-17); Lyons cannot search for the main object to an infinite distance from the 
camera; thus, the searching range must be restricted in that respect. Secondly, whatever is 
outside of the camera's field of view cannot be searched at any depth.) for searching said main 
subject (21) from the plurality of objects (21,21a, 22, and 23) using the input image data ; and 
. a partial image extracting unit (55) for extracting a partial image ("central region 26" or 
"peripheral region 27") including said main subject (21) from said image data, a depth direction 
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of said partial image being restricted to said searching range set by said range setting unit (see 
column 7, lines 55 - 65, column 8, line 11 - 22, and column 10, lines 46 - 62); and 

a main subject detecting unit (15/222) for detecting a main subject (21) from said partial 
image ("central region 26") and receiving main subject information (see column 7, line 55 - 
column 8, line 22), wherein 

wherein said image data input unit comprises a means for inputting a parallactic image 
(see figure 9), which picks up an image of the plurality of objects (21,21a, 22, and 23) from a 
plurality of different perspectives, and the image processing apparatus (222) further comprises a 
depth distribution information receiving unit for receiving said depth distribution information 
based on said parallactic image (28). 

Lyons discloses a tracking systems that first searches for the main object in the a narrow 
field of view and then expands the search to a wide angle field of view if the main object leaves 
the narrow field of view. Lyons proposes a series of systems to accomplish this task — the 
system in figure 1 A uses a special type of optics where both fields of view are nearly aligned and 
the system in figure 9 uses a two-camera system to input a parallactic image. The main object is 
searched for in either narrow angle image or the wide angle image; however, both images 
include a same spot. While Lyons discloses restricting the searching range to a single 
predetermined distance, either a depth of field corresponding to a "wide-angle" FOV 27 or a 
"narrow-angle" FOV 26, in the depth direction for search a main subject from the plurality of 
objects; 

Lyons is silent with respect to extracting the main subject based on depth distribution 
information indicating a distance to each of said plurality of objects included in said image data. 
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On the other hand, Anderson also discloses an image processing apparatus including a 
range setting unit and an image extracting unit for extracting said main subject from within the 
set searching range. Specifically, Anderson teach, as shown in figures 1, 3 A, 3B, 4B, an image 
processing apparatus including a range setting unit and an image extracting unit for extracting 
said main subject (e.g., 504) from among a plurality of objects (e.g., 502, 508, and 506) from 
within the set searching range (500). Anderson further teaches, as shown in figure 7 and as 
stated in column 8 (line 1) - column 9 (line 57), extracting the main subject (504) from the 
image data (500) based on depth distribution information (see figure 6A) indicating a distance to 
each of said plurality of objects included in said image data (see column 8,lines 1-13). 

Hence, at the time invention was made, it would have been obvious to one with ordinary 
skill in the art to have included the extracting the main subject from the image data based on 
depth distribution information indicating a distance to each of said plurality of objects included 
in said image data (as taught by Anderson) in the image processing apparatus (disclosed by 
Lyons) for the advantage of reducing undesirable image characteristics and improve image 
quality (see column 1, lines 61 and 62). 

7. As for Claims 5 and 23, Lyons discloses, as shown in figures 9, 2A, 2B, and 8B and as 
stated in column 10 (lines 46 - 62), said range setting unit (15) comprises a means for setting a 
first searching range ("narrow-angle field of view" 26) and a means for setting a second 
searching range ("wide-angle field of view" 27), which differs from said first searching range 
(see column 5, lines 61 - 67); 

said partial image extracting unit (15/222) comprises a means for extracfing the part of 
the plurality of objects (21,21a, 22, and 23) included in said first searching range (Step S91) as a 
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partial image and a means for extracting the part the plurality of objects (21, 21a, 22, and 23) 
included in said second searching range (Step S93) as a second partial image; and 

said main subject detecting unit (15/222) comprises a means for detecting (Step S91) said 
main subject (21) from said first partial image and a means for detecting (Step S93) said main 
subject (21) from said second partial image when said main subject is not detected from said first 
partial image (see figure 8B). 

8. As for Claim 6, Lyons discloses, as shown in figure SB, wherein said range setting unit 
(15) sets a predetermined depth length ("narrow-angle field of view") as said first searching 
range and sets the depth length contiguous (see column 7, line 55 - column 8, line 22; and 
column 10, lines 46 - 62) to said first searching range as said second searching range. 

9. As for Claims 7, 8, 24, and 25, Lyons discloses as stated in column 5 (lines 52 - 57), 
wherein said range setting unit (15) sets the depth length ("narrow-angle field of view"), which 
includes an independent object (21) existing at the nearest distance (i.e., zoomed-in), as said first 
searching range based on a depth distribution information (see column 5, lines 52 - 57) and sets 
the depth length contiguous to said first searching range as said second searching range (see 
figure 8B). 

10. As for Claim 9, Lyons discloses, as indicated above, said range setting unit (15) sets a 
different said searching range (Step S93) when said main subject is not detected (Step S92) in 
said searching range set by said range setting unit (15); said partial image extracting unit extracts 
again (Steps S91) said partial image; and detects again said main subject. 

11. As for Claims 10 and 26, Lyons discloses, as shown in figures 8B and as stated in 
column 10 (lines 46 - 62), wherein said image data input unit is an image capturing unit (17/51) 
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for picking up an image of the plurality of objects (21,21a, 22, and 23), comprising: a 
photographic condition setting unit (15) for setting a photographic condition (zoom condition) 
based on said main subject information; and an image capturing control unit for controlling 
imaging by said image capturing unit based on said photographic condition (see figure 8B). 

12. As for Claims 14 and 30, Lyons inherently discloses wherein said image data input unit 
is an image capturing unit, which captures an image of said object based on a photographic 
timing signal. However, Lyons is silent with respect to setting a photographic timing condition 
relating to the main subject, wherein said photographic timing is predetermined and stored. 

However, Official Notice (MPEP § 2144.03) is taken that both the concepts and 
advantages of providing a timing condition storing unit for storing a predetermined photographic 
timing condition relating to said main subject; and a timing signal output unit for outputting said 
photographic timing signal to said image capturing unit when said main subject satisfies said 
photographic timing condition are well known and expected in the art. At the time the invention 
was made, it would have been obvious to one with ordinary skill in the art to have a timing 
condition storing unit for storing a predetermined photographic timing condition relating to said 
main subject; and a timing signal output unit for outputting said photographic timing signal to 
said image capturing unit when said main subject satisfies said photographic timing condition for 
the advantage optimizing exposure and dynamic range of the main subject. 

13. Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Lyons in view of 
Anderson in further view of Hanna et al. (US 6 714 665 Bl). 
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14. As for Claim 4, Lyons (as modified by Anderson) discloses a main subject detecting unit 
(15/222) for detecting a main subject (21) from said partial image ("central region 26") and 
receiving main subject information. 

However, Lyons (as modified by Anderson) does not disclose a distinctive parts detecting 
unit for detecting from said partial image, a distinctive part, which should be included in said 
main subject; and an information receiving unit for receiving said main subject information 
based on the position of said distinctive part detected by said distinctive parts detecting unit. 

On the other hand, Hanna et al. also disclose an image processing apparatus including an 
image data input unit, a range setting unit, and a partial image extracting unit. Mores 
specifically, Hanna et al. teach, as shown in figures 1 A and IB, a distinctive parts detecting unit 
(computer 20) for detecting from said partial image (NFOV camera 14), a distinctive part 
("user's eye" - see figure IB), which should be included in said main subject; and an 
information receiving unit (18) for receiving said main subject information based on the position 
of said distinctive part detected by said distinctive parts detecting unit (see column 5, lines 40 - 
51 and column 7, lines 17 - 30). 

As stated in column 1 (lines 50 - 53) of Hanna et al., at the time the invention was made, 
it would have been obvious to one with ordinary skill in the art to have included a distinctive 
pars detecting unit and an information receiving unit (as taught by Harma et al.) in the image 
processing apparatus (disclosed by Lyons - as modified by Anderson), for the advantage 
"identifying objects of individuals in a passive way that is both fast and accurate." 
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Conclusion 

15. Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Justin P Misleh whose telephone number is 571.272.7313. The 
Examiner can normally be reached on Monday through Friday from 8:00 AM to 5:00 PM. 

If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Lin Ye can be reached on 571.272.7372. The fax phone number for the organization 
where this application or proceeding is assigned is 571.273.8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

/Justin P. Misleh/ 
Examiner, GAU 2622 
January 17, 2008 



